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Abstract 

Since 2003, international project PISA evaluates 15-year old students in solving problems that include “decision 
taking”, “analysis and design of systems” and “trouble-shooting”. This article presents the results of a pilot research 
conducted with 215 students from first to fourth grade of a high school in Sarajevo (Bosnia and Herzegovina). The 
students, in an imaginary, real-life like scenario, had to advice a bus company about number of buses needed for 
operating a 24-hour service between two cities. Research instrument was structured in a way, which permits 
exploring how students deal with a “complex problem” (in PISA terminology it would be problem type “analysis 
and design of systems”) in real-life setting. Negative research results show that students mainly (1) use either verbal 
or visual reasoning mode; (2) have poor “sense-making” approach in analyzing a simple dynamical system and (3) 
have underdeveloped an important competency for working in knowledge-based economy (decision-taking based on 
evidence and arguments). A positive result is that majority of students found interesting the problem in question. 
Some brief recommendations about the presence and role of “complex problem” in physics teaching are offered in 
conclusions. 

Keywords: Complex problem solving, reasoning modes, decision taking. 


Introduction 

The "knowledge-based economy" (OECD, 1996; Cook, 2001) and "learning companies" (Senge, 
1990; Marquardt & Reynolds, 1994) depend critically, and each day more, of the workforce that 
is capable to learn and use creatively existing knowledge or to build new knowledge in order to 
address in a novel way challenges posed by rapid changes in the market (Nonaka & Takeuchi, 1995). 
The most appreciated feature of this labor force are the willingness and the skills for continuous 
learning throughout life, so called “life-long learning”. 

It is self-understood that the quality and speed of enterprise learning, which are today the 
only confident competitive advantage of any company, can not be achieved, in an organized and 
efficient way, without having the "knowledge workers" able to identify, discuss and resolve 
critical business problems. The higher education institutions have a responsibility to prepare 
these workers, not only for current and potential problems of today but more for those that will 
emerge in the coming decades (Jarvis, 2001; Graham, 2002). 

Nevertheless, emerging problems in developing novel production technologies or selling 
strategies are rather complex because treated systems have many related parts with multiples 
causal connections and numerous operational restrictions. Being so, it would be unwise and 
shortsighted to believe that only universities can prepare “complex-problem solvers”. Of course, 
every persons need these skills not only to carry out successfully a profession (or to change it) 
but also to face many problems in personal and social life. Everybody needs abilities to make the 
best decisions when buying things, renting a car or considering loans plans. 

Education system as a whole must have as its basic aim to create multiple and age-tuned 
learning opportunities for students to get “knowledge and skills for life”. Such vision is the base 
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of the project PISA, which (since 2003) evaluates how 15-year old students deal with problems 
like “decision taking”, “analysis and design of systems” and “trouble shooting” (PISA, 2003). 
Those students who are good at solving such types of problems are on a right road to become 
efficient “complex-problem solvers”. 

Trends in problem designs in physics education 

Since two decades ago, among those, who sustain the constructivist positions on the design 
of learning environments exist a broad consensus that it is preferable to use "real-world problems" 
(Jonassen, 1991) or "real problems" (Wilson and Cole, 1991) as one of the key elements to encourage 
the construction of meaningful learning. 

In physics education, this consensus is reflected in so called „context-rich problems 14 
(Heller, Keith & Anderson 1992, Heller & Hollabaugh, 1992). This particular name was chosen 
to stress the difference with “context-poor-problems” which are formulated with little or none 
relation to the real-world situations. Due to their relative openness, they might be called 
“complex problems”, too. “Context-rich-problems” are formulated in everyday life contexts in 
which: 

(1) default unknown variable need not to be explicit, 

(2) there may be more information available than necessary for their solution, 

(3) some information may be missing, and which require reasonable assumptions to 
simplify the problem situation and enable meaningful solutions. 

Students' performances with „context-rich problems were explored in many experimental 
studies which reported variable results (Huffman, 1997; Yerushalmi & Magen, 2006; Enghag, 
Gustafsson & Jonsson, 2007, Walsh, Howard & Bowe 2007, Enghag, Gustafsson & Jonsson 
2009). Some research reports indicate that most students cannot independently solve such 
problems and claim that problem solving skills should be an explicit element of teaching. In 
addition, “context-reach problems” are mainly used at the university level, leaving pre-university 
physics teaching and learning dominated by traditional “context-less problems”. 

A recent study in Croatia (Erceg, Marusic & Slisko, 2011) shows that less than 2 of 10 pre¬ 
university students are able to deal correctly with “partially specified physics problems” (Slisko, 
2008), which are fonnulated in the way that students have to decide what to calculate and how to 
evaluate obtained results. In that reference, one can find a more detailed account of old and 
recent trends in physic problem designs. 

The aims, problem design, methodology of data collecting and expected results 

The aims of this research were to find out initial answers to the following research 
questions: 

(Ql) How do students from a high-school in Bosnia and Herzegovina solve a PISA 
problem type „design and analysis of system", presented in real-life setting? 

(Q2) How much are students aware of the complexity and openness of the problem and 
which additional data they would required? 

(Q3) How do they accept or reject a different solution of the problem and which arguments 
they use to support their decision? 

(Q4) How do they compare this type of problem with the types of problems they normally 
face in school? 


39 



EJPE 


European Journal of Physics Education Volume2 Issue 3 2011 ISSN 1309 7202 Balukovic et al 


After an introductory narrative which create a real-life scenario and setting, these research 
questions are implemented in the following way (Box 1): 

Box 1. Worksheet students worked with in the research _ 

How many buses are needed? 

A transportation company is considering a project to establish a every-full-hour, day-and-night bus service 
between two cities A and B. 

Every full hour a bus would leave the city A for the city B and one from the city B to the city A. Both travels, 
from the city A to the city B and from B to the center of A, would take exactly 3 hours. 

Imagine you are one of the advisers in this company that wants to have an expert answer, as accurate as possible, to 
the question: What is the number of buses needed to establish efficient, two-way, permanent connections 
between city A and city B, during 24 hours? 

Director of this company has also presented to you the answers of other advisors. Your task is to accept or turn 
down each of these responses and, of course, to argument your decision with as many details as possible. 

Part A. Before your start assessing the responses of other advisers, try to find your own answer to the question 
about the number of buses needed. 

1. Your answer is; "It is necessary to_buses". 

Argumentation: 


2. If in your way of thinking, it was necessary to visually present functioning of the future connection between the 
city A and the city B, then draw your image of the connection and describe by words all parts of your drawing. 


Part B. If for a precise answer you need some missing data, list three most important of them and argument why 
and how they would help. 

1. Data about _ 

is needed for_ 

2. Data about _ 

is needed for_ 

3. Data about _ 

is needed for_ 

Part C. Other consultants’ responses, you are supposed to evaluate carefully, were: 
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1. It takes 6 buses. 

(a) Accept, (b) Reject, (c) I cannot decide. 
Arguments_ 

2. It takes 7 buses. 

(a) Accept, (b) Reject, (c) I cannot decide. 
Arguments 


3. It takes 8 buses. 

(a) Accept, (b) Reject, (c) I cannot decide. 
Arguments 


4. It takes 9 buses. 

(a) Accept, (b) Reject, (c) I cannot decide. 
Arguments 


5. The number of buses is not possible to determine from available data. 

(a) Accept, (b) Reject, (c) I can not decide. 

Arguments 


Part D. One of these advisers said: "I found this problem much more interesting than any problems that I dealt 
with it school." What is your attitude to that opinion? 

(a) Strongly agree, (b) Partially agree, (c) I have no opinion. 

(d) Partially disagree, (e) Strongly disagree. 

Arguments 


As it is easy to note, the Part A is related to the research question Q1, the Part B with the 
research question Q2, the part C with the research question Q3 and the Part D with the research 
question Q4. Actual work sheet is a substantial expansion of a previous one in which only one 
question was asked: What is a minimal number of buses needed to operate smoothly this bus 
connection? (Pecina & Slisko, 2007). 

The above-presented work sheet was fdled up anonymously by 215 participating high- 
school students (II Gimnazija, Sarajevo, Bosnia and Herzegovina). The survey was conducted in 
the school year 2010/2011. Distribution of students according their grade was as follows: 47 
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fourth-grade students of mathematics and natural science section, 56 third-grade students of 
mathematics and natural sciences section, 50 second-grade students and 62 first-grade students. 
Students’ ages ranged from 15-year to 19-year old. 

It was expected that students should be able 

(1) to find out that the number of on-road buses must be eight, and 

(2) to estimate number of “reserve” buses, taking into account known or asked additional 
data. 

Of course, the most “saving” solution would be to have one or two “reserve” buses at the 
stations in the cities A and B. So, an initial realistic answer would be that 10 or 12 buses are 
needed. 

Students’ answers and reasoning modes in the Part A 

According to their answers in the Part 1, the students were divided into seven different groups, 
shown in the Table 1. 


Categories 

Students’ Answers (number of buses) 

I 

n<6 

II 

n=6 

III 

n=7 

IV 

n=8 

V 

n>8 

VI 

impossible to determine the number of buses 

VII 

no answer 


Table 1. Categorization of the students ’ answers in the Part A 

Group category II can be divided into two distinguished subgroup: (1) Students who 
mention that the answer refers to an ideal situation: assuming zero time between arriving and 
leaving; (2) Students who simply conclude that for the 2 parallel bus lines in 3 hours is equal 6 
bus rides for 6 buses. 

Category Group IV also has three subgroups: (1) Students who have “calculated” that 8 
buses is a right solution, in a “simply way” that 24 - hours divide by 3 hours is 8; (2) Students 
who apply a time based criteria considering that each bus travels for 3 hours between the two 
towns and three such tours per day (24 hours), which gives a simple account of 8 buses; (3) 
Students who had a vision of spatial problem solutions getting the essential system configuration 
in their account. 

The resulting answers of students to the problem-solving task in Part A for each category 
is shown in table 2. 


Category 

Number of students 

(%) 

I 

2 

0,9 

II 

144 

67 

III 

3 

1,4 

IV 

42 

19 


42 











European Journal of Physics Education Volume2 Issue 3 2011 ISSN 1309 7202 Balukovic et al 


EJPE 


V 

9 

4,2 

VI 

2 

0,9 

VII 

14 

6,5 


Table 2. Distribution of student answers by categories 


Only eight buses are needed 

Out of 42 student who answered that 8 buses were necessary - 15 students (35.6%) gave 
their answers without an explanation, while 23 students (56.1%) gave superficial answers. 
For instance, 21 students drew erroneous conclusions being guided by the following 
argumentation: 

The city A and the city B are 3h distance from one another, and the question is how many buses 
are necessary during 24 h. If one bus leaves every hour, so that two-way connection is 
established, then it is necessary 24 h : 3 h = 8, which means that a total of 8 buses are necessary, 4 
from the city A and 4 from the city B. 

Only 4 students (7.3%) gave a correct answer for necessary number of on-road buses. Two of 
them got the answer using only visual reasoning mode (Figure 1 and Figure 2). 



Figure 1. Abstract visual mode of reasoning (buses represented by numbers). 
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Figure 2. Concrete visual mode of reasoning (buses represented by a realistic icon). 

The third student used purely verbal reasoning: 

If the first drive starts at 01:00 o’clock, the buses 1 and 2 will be needed. 

At 02:00 o’clock, the buses 3 and 4 will be needed. 

A 03:00 o’clock, the buses 5 and 6 will be needed. 

At 04:00 o’clock the bus 7 will leave the point A, while the bus 8 will leave the point B. 

The bus 1 will arrive to the point B at 04:00 o’clock, but it will leave at 05:00 
o’clock due to fueling and cleaning. The same happens to the bus 2, so these 8 buses will 
be circulating. 

It is important to note that all three students used starting-from-zero-time approach to get 
to the essential system configuration, which permits easily, counting of minimal number of buses 
in regular system operation. This step-by-step reasoning is characteristic behavior, when students 
are dealing with an u nkn own or difficult problem 

The fourth students got 8 buses by using an amazing (although not completely correct) 
real-life consideration. The start is a little bit unclear. One can only guess that the students try to 
reject the answer “6 buses”: 

As we have 4 points inside the interval 0 h - 3 h, 24:4= 6, and every point must be filled. This 
result is possible only in ideal conditions, while fueling, possible malfunctions and traffic stops are 
not considered 

After that introduction, the student offers this reasoning: 

According to the European Union standards, a driver of passengers-carrying vehicle is allowed to 
drive up to 9 hours, after that he must pause for 6 hours. One driver may drive this distance three 
times. Regular bus consumption is about 20 liters on 100 km, and mean speed is 75 km/h. S = 3 h * 
75 km/h = 225 km. For 225 km (the bus) uses 45 liters of fuel. The tank capacity is 100 liters, for 6 
hours it spent 90 liters of fuel and then (the driver) must fuel and fueling takes at least 30 minutes. 
During this 30 minutes, bus cleaning follows, too, and the bus would have a 30minute delay every 
6 hours. 24:6 = 4 delays. 4 * 30 minutes = 120 minutes = 2 hours. Two more buses should be 
introduced, it means 8 buses are needed. 
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This example of fact-based and simple-physics reasoning shows that students kn ow much 
more things about real-world than teachers cannot even imagine. They deserve chance to show 
what they know and complex-problem solving is certainly a mode to do it. 

Only six buses are needed 

Out of 144 students who answered that 6 buses were necessary, 59 students (41%) gave 
their answer without an explanation. 85 students (59%) gave superficial answers, being 
guided by the following “logic”: 

Considering that every hour a bus leaves from city A to city B and vice versa, that means that 2 buses 
leave every hour. Since we have 3 hours, a total number of buses would be 2*3= 6 buses, or 6 buses 
are needed - 3 from each city, that would make 4 rounds on their routes during the entire day. (i.e. 24 
h :4 hours, which represents the distance between the cities, totals 6 buses by day.) 

By a more detailed analysis, it has been established that 38 students, without noticing that 
their way of reasoning is based on an unrealistic assumption about “zero time” between arriving 
and leaving (see below a comment about this phenomenon): 

First bus left at midnight, after lh second bus leaves at lh, and then it comes back to the city A 
for 6h i.e. at 7h, so the line is: l-7h, 7-13h, 13-19h, 19-lh; The third one leaves at: 2-8h, 8-14h, 
14-20h, 20-2h; Fourth one: 3-9h, 9-15h, 15-21h, 21-3h; Fifth one: 4-10h, 10-16h, 16-22h, 22-4h; 
Sixth: 5-1 lh, 11:17h, 17-23h, 23-5h, an then the first one comes back at 6 h. 6 buses are 
necessary. 

If a bus leaves from city A to city B every hour, and also from city B to city A, that means that 2 
buses leave every hour. Since there is three hour distance between 2 cities, and 2 buses leave 
every hour, which means that the number of buses is 3*2=6 buses. 


2 at the begining at 0 hour two buses starts, one from A and second from B 
+2 at 1 hour two more buses start, one from A and second from B 
+2 at 2 hour two more buses start, one from A and second from B 

= 6 at 3 hour two first buses arive, first in B and second in A and from there they start the second 
circle. 

As in the case of 8 buses, some students supported their answers in different ways via drawings. 
Some of them used starting-from-zero-time approach to get the answer (Figure 3): 
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Figure 3. Hour-after-hour approach to getting essential configuration of the system with numerical 

representation of buses. 

Nevertheless, differently from above, there are students who didn’t start from the zero 
time, but captured the situation “frozen at one arbitrary hour” (Figure 4 and Figure 5): 
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Figure 4. A student’s drawing used for getting answer “6 buses” with numerical representation 

of buses. 



Figure 5. A student’s drawing used for getting answer “6 buses” with iconic representation of 

buses. 

Some students felt necessity to “cover” all 24 hours of bus service operation. Below 
(Figure 6 and Box 2), an original time-table and its transcript are shown. 
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Figure 6. 24-hour time-table presented by one student. 


Box 2. The transcript of the time-time in the Figure 6. 


The hour when the buses start 

A 

B 

22 

19 

16 

13 10 

7 4 1 

1 

1 

23 

20 

17 

14 11 

8 5 2 

1 

1 

24 

21 

18 

15 12 

9 6 3 

1 

1 


24 hours buses = 6 


It is important to mention that some students needed concrete, familiar names of the 
cities, like Bosnian cities Tuzla and Sarajevo, in order to lay down their reasoning (Figure 7): 
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Figure 7. Contextualization of the problem with familiar names of cities Tuzla and Sarajevo 

(instead of A and B). 

This is an important message for physics problem designers: There are students who are 
unable to think about solving physics problems that are formulated in abstract contexts. They 
would perfonn much better if familiar contexts are provided. Some of them might do 
“contextualization” by themselves, but some of them are lost. The same happens with times in 
this problem with buses. Namely, as was see above, some choose concrete full-hour times in 
their reasoning, but some say that the problem can’t be solved because no concrete leaving times 
are given. 

Coherence between verbal and visual reasoning and the need to use both 
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Generally speaking, students involved in this research show asymmetric use of reasoning 
modes, either verbal (mainly) or visual. For example, very often verbal reasoning is not followed 
by a visual representation or presented visual representation of the problem situation is rather 
primitive (only cities A and B with a 3-hour “distance”). The same can be said for those students 
who prefer visual reasoning. 

Very few students used both reasoning mode in a coherent way. One example is shown in the 
Box 3 and in the Figure 8. 

Box 3. Verbal reasoning of a student who used coherently two reasoning modes. _ 

Hour 0: Buses 1 and 4 at the stations 

Hour 1: Buses 1 and 4 in motion, buses 2 and 5 at the stations 

Hour 2: Buses 1, 2, 4 and 5 in motion, buses 3 and 6 at the stations 

Hour 3: Buses 6, 5, 3 and 2 in motion, buses 1 and 4 at the stations. 
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Figure 8. Visual reasoning of a student who used coherently two reasoning modes. 


According to Gardner’s theory of multiple intelligences (Gardner, 1983; Gardner & Hatch, 
1989), it is quite normal that there are differences in the grade of usage of verbal and visual 
intelligences. Nevertheless, in physics teaching and learning it is very important to cultivate both 
of them (van Heuvelen, 1991; Dufresne, Gerace & Leonard, 1997). Namely, very few physics- 
problem situations are not prone to be represented visually. On the other side, many physics 
problems, as well as mathematics problems, can’t be solved without having drawing problem 
situation or corresponding graph. 

In fact, recently many mathematics educators recognize a crucial role of the construction of 
the “situation model” in the process of problem solving (Nesher, Hershkovitz & Novotna, 2003; 
Thevenot et al., 2007; Brissiaud & Sander, 2010). Some results show that students use visual 
solving methods when dealing with novel and difficult problems and try non-visual strategies in 
less difficult situations (Lowrie & Kay, 2001). 


The presence of the known phenomena of “suspension of sense-making’’ 
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In mathematics education, there are many experimental evidences that students, when 
solving problems do not make sense of the obtained results even if they have necessary real- 
world knowledge to do so. This phenomenon is kn own as “suspension of sense-making (Palm, 
2008; Verschaffel et ah, 2010). We found that phenomenon widely present in the “six buses” 
answers of the students. 

Namely, 106 students reached this answer without even try to make sense of it: 

6 buses, in order to establish constant link with the conditions mentioned above, 6 buses 
meet all criteria. 

6 buses necessary because it takes 6 h for each bus to come back to the destination and to 
set-off again. 

Very rarely, students state clearly “ideal supposition” this answer is based on: 

At the same time two buses are leaving, one from the city A and the other from the city B. 
After one hour they completed 1/3 of the travel and two new buses must leave. After one 
more hour, first two completed 2/3 of the travel and second two 1/3 of the travel. After 
one more hour, first two are entering A and B, so they go back. The same continues with 
the next two after one more hour. So, we have used 6 different buses. This happens only 
in ideal conditions (passengers set in and out in the same moment) . 

“Six buses in ideal conditions, if we take into account that the bus immediately leaves the 
city it arrived in . ” 

Even these students, who are aware of what their results imply, do not show any need to 
reject it and look for more realistic answer. One possible reason is that they believe that physics 
problem solving a game applicable only for ideal situations, having little or nothing in common 
with a real-life. 

Students’ results in the Part B 

From a total of 215 students, only 9 (4.2 %) students thought that they needed additional 
information to be able to answer how to solve the given problem (Part A). Five students stated that they 
need 3 additional information for solving the Part A of the given problem, 3 students needed 2 new 
information and one student would like to receive only one new detail as is shown on the Figure 9. 
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Figure 9. Distribution of students according additional information the asked for. 


The required additional information are different: bus schedule, time for breaks or time 
spent waiting at the station after the ride, fuel tank size, number of passengers on the bus, the 
physical condition of the bus-driver, bus length, the number of stops on the route between two 
towns. 

It is significant to note that among the 14 (6.5%) students who do not respond in part A 
was not even a student who would need new information that could solve the problem. 


Students’ results in the Part C 

Each student had the possibility to accept or not to accept the answers given by advisors 
working for that company. An interesting fact is that out of 215 students, everyone agreed with 
the advisor who gave the same answer, which the students reached via visual or verbal 
reasoning. Students, whose answer did not match any of advisor’s that gave specific answer, 
chose the one that claimed that the number of buses could not be found from the data available. 

Explanations for decision made was provided by 25 (11,6%) students: 

• 17 students accepted the answer given by one of the advisors 

• 7 students rejected the answer given by one of the advisors 

• 1 student could not decide on the solution given by any of the advisors 

Eight students provided explanation for accepting the answer without any argumentation. 
Explanation examples are: «it is consistent with my answer», «because it is needed)), or 
«advisors proposed answers that matched my own». Conversely, 9 students provided more 
precise explanations, together with sustaining arguments. Explanations examples of those 
students are: 

I accept 8 buses as the solution because it is the only option. I have rejected 6 buses 
because I find it impossible that there would be no traffic jams, delays or station problems. 

I think it is impossible to make estimation out of these data, because we do not know the 
timetables. One or two minutes earlier or later completely changes the situation. 

There is no sufficient data to solve the problem because i.e. buses have to fill their tanks. 
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I accept the answer: 6 buses, but only in ideal conditions (no refills, no flat tires etc). 

Rejection of one of the answer given by advisors was partially provided in two following 

cases: 

I reject the advisors’ answer: 6 buses, because it is not realistic - there would be no breaks 
between the drives. 

I reject 9 buses, although I think it is better to have one vehicle more because of the traffic jams 
or defects. 

The explanation for not making a decision was provided by one student, who thought 
there were no sufficient data to solve the problem, since situations like defects have not been 
taken in consideration. 

It is evident that almost 90% of students lack the culture of providing a substantially 
argumentative answer to a problem. This finding should be carefully considered together with 
the fact that almost half of these students are just one year away from entering universities, 
where they will be faced with more complex problems to solve. 

Students’ results in the Part D 

Regarding the question whether the given problem was more interesting than any of the 
problems that they had been offered to resolved at the school, the results are as shown in the 

Figure 10. 



agree agree disagree disagree 


Figure 10. Students ’ responses in the Part D in which evaluate statement the “bus problem ” is more 
interesting than problems they usually face in the school. 

According to the responses received by all students, one can conclude that 126 students (58.5%) 
do agree that this problem is more interesting than any problems solved before in the school. 
Disagreement showed 45 students (21%). From these data we are please to infer that the 
problems of this type should be more present in schools to improve the quality of teaching, and 
consequently the interest of students for the practical problems related to their real-life 
environment. 


51 










EJPE 


European Journal of Physics Education Volume2 Issue 3 2011 ISSN 1309 7202 Balukovic et al 


Conclusions 

This research shows that high-school students are no able to deal with relatively simple complex 
problem, even on its basic level related to the number of on-road buses. The most popular answer 
of six buses is reached supposing, implicitly or explicitly, an “ideal situation” in which a bus 
arrives and leaves the station in the same moment. Accepting this idea is another example of 
“suspension of sense-making” in solving school problems. In physics learning it might be a 
natural outcome of dealing with to many idealizations (frictionless motions, massless string or 
massless spring, point masses o point charges,...). 

Students show that they are mainly satisfied with the first answer they are able to find. In 
consequence, they rarely asked for more data (Part B) or reconsidered it when knowing other 
answers (part C). They accept easily other advisers’ answer only if it is equal to theirs and reject 
all different ones. The problem with rejection is that decisions are not supported by any 
argument. Such closeness regarding other options is not a feature they should be proud of. 

The only positive result of this research is that students find the problem in question more 
interesting that other school problems. 

Taking into account what was said above, we think that “complex problems” should be 
more present in high-school physics textbooks and classroom. “Context-rich” or “partially 
specified” physics problems might be first options to try. In addition, different reasoning modes, 
with all their strengths and weakness, should be permanently cultivated. As Feynman said, we 
know more if we solve one problem in three different ways than if we solve three different 
problems in only one way. 
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